policy analysis
The Longitudinal Health, Income, and Employment Model (LHIEM): a discrete-time microsimulation model for policy analysis
Propp, Adrienne M., Vardavas, Raffaele, Price, Carter C., Kapinos, Kandice A.
Dynamic microsimulation has long been recognized as a powerful tool for policy analysis, but in fact most major health policy simulations lack path dependency, a critical feature for evaluating policies that depend on accumulated outcomes such as retirement savings, wealth, or debt. We propose the Longitudinal Health, Income and Employment Model (LHIEM), a path-dependent discrete-time microsimulation that predicts annual health care expenditures, family income, and health status for the U.S. population over a multi-year period. LHIEM advances the population from year to year as a Markov chain with modules capturing the particular dynamics of each predictive attribute. LHIEM was designed to assess a health care financing proposal that would allow individuals to borrow from the U.S. government to cover health care costs, requiring careful tracking of medical expenditures and medical debt over time. However, LHIEM is flexible enough to be used for a range of modeling needs related to predicting health care spending and income over time. In this paper, we present the details of the model and all dynamic modules, and include a case study to demonstrate how LHIEM can be used to evaluate proposed policy changes.
Using LLMs for Automated Privacy Policy Analysis: Prompt Engineering, Fine-Tuning and Explainability
Chen, Yuxin, Tang, Peng, Qiu, Weidong, Li, Shujun
Privacy policies are widely used by digital services and often required for legal purposes. Many machine learning based classifiers have been developed to automate detection of different concepts in a given privacy policy, which can help facilitate other automated tasks such as producing a more reader-friendly summary and detecting legal compliance issues. Despite the successful applications of large language models (LLMs) to many NLP tasks in various domains, there is very little work studying the use of LLMs for automated privacy policy analysis, therefore, if and how LLMs can help automate privacy policy analysis remains under-explored. To fill this research gap, we conducted a comprehensive evaluation of LLM-based privacy policy concept classifiers, employing both prompt engineering and LoRA (low-rank adaptation) fine-tuning, on four state-of-the-art (SOTA) privacy policy corpora and taxonomies. Our experimental results demonstrated that combining prompt engineering and fine-tuning can make LLM-based classifiers outperform other SOTA methods, \emph{significantly} and \emph{consistently} across privacy policy corpora/taxonomies and concepts. Furthermore, we evaluated the explainability of the LLM-based classifiers using three metrics: completeness, logicality, and comprehensibility. For all three metrics, a score exceeding 91.1\% was observed in our evaluation, indicating that LLMs are not only useful to improve the classification performance, but also to enhance the explainability of detection results.
A Multi-LLM-Agent-Based Framework for Economic and Public Policy Analysis
We first evaluate five LLMs' economic decision-making capabilities in solving two-period consumption allocation problems under two distinct scenarios: with explicit utility functions and based on intuitive reasoning. While previous research has often simulated heterogeneity by solely varying prompts, our approach harnesses the inherent variations in analytical capabilities across different LLMs to model agents with diverse cognitive traits. Building on these findings, we construct a Multi-LLM-Agent-Based (MLAB) framework by mapping these LLMs to specific educational groups and corresponding income brackets. Using interest-income taxation as a case study, we demonstrate how the MLAB framework can simulate policy impacts across heterogeneous agents, offering a promising new direction for economic and public policy analysis by leveraging LLMs' human-like reasoning capabilities and computational power.
Machine Learning-Based Security Policy Analysis
Jain, Krish, Sum, Joann, Kapoor, Pranav, Eaman, Amir
Security-Enhanced Linux (SELinux) is a robust security mechanism that enforces mandatory access controls (MAC), but its policy language's complexity creates challenges for policy analysis and management. This research investigates the automation of SELinux policy analysis using graph-based techniques combined with machine learning approaches to detect policy anomalies. The study addresses two key questions: Can SELinux policy analysis be automated through graph analysis, and how do different anomaly detection models compare in analyzing SELinux policies? We will be comparing different machine learning models by evaluating their effectiveness in detecting policy violations and anomalies. Our approach utilizes Neo4j for graph representation of policies, with Node2vec transforming these graph structures into meaningful vector embeddings that can be processed by our machine learning models. In our results, the MLP Neural Network consistently demonstrated superior performance across different dataset sizes, achieving 95% accuracy with balanced precision and recall metrics, while both Random Forest and SVM models showed competitive but slightly lower performance in detecting policy violations. This combination of graph-based modeling and machine learning provides a more sophisticated and automated approach to understanding and analyzing complex SELinux policies compared to traditional manual analysis methods.
Privacy Policy Analysis through Prompt Engineering for LLMs
Goknil, Arda, Gelderblom, Femke B., Tverdal, Simeon, Tokas, Shukun, Song, Hui
Privacy policies are often obfuscated by their complexity, which impedes transparency and informed consent. Conventional machine learning approaches for automatically analyzing these policies demand significant resources and substantial domain-specific training, causing adaptability issues. Moreover, they depend on extensive datasets that may require regular maintenance due to changing privacy concerns. In this paper, we propose, apply, and assess PAPEL (Privacy Policy Analysis through Prompt Engineering for LLMs), a framework harnessing the power of Large Language Models (LLMs) through prompt engineering to automate the analysis of privacy policies. PAPEL aims to streamline the extraction, annotation, and summarization of information from these policies, enhancing their accessibility and comprehensibility without requiring additional model training. By integrating zero-shot, one-shot, and few-shot learning approaches and the chain-of-thought prompting in creating predefined prompts and prompt templates, PAPEL guides LLMs to efficiently dissect, interpret, and synthesize the critical aspects of privacy policies into user-friendly summaries. We demonstrate the effectiveness of PAPEL with two applications: (i) annotation and (ii) contradiction analysis. We assess the ability of several LLaMa and GPT models to identify and articulate data handling practices, offering insights comparable to existing automated analysis approaches while reducing training efforts and increasing the adaptability to new analytical needs. The experiments demonstrate that the LLMs PAPEL utilizes (LLaMA and Chat GPT models) achieve robust performance in privacy policy annotation, with F1 scores reaching 0.8 and above (using the OPP-115 gold standard), underscoring the effectiveness of simpler prompts across various advanced language models.
PhD Position In Artificial Intelligence At Delft University Of Technology 2021-2022 - AI Summary
The importance of considering distributive justice in climate policy motivates research in AI-based decision support to search for balanced alternatives across multiple sectors, regions, and generations and counteract existing asymmetries in policy design. This PhD position is one of the four PhD positions in the Hippo Lab (Hyper-heuristics for interpretable public policy analysis), which is part of the TU Delft Artificial Intelligence initiative to channel expertise in AI foundations to tackle societal and scientific challenges. With its excellent education and research at the intersection of technology, society and policy, the Faculty of TPM contributes to solving complex technical-social issues, such as energy transition, mobility, digitalisation, water management and (cyber) security. Stay updated on last news about Artificial Intelligence. Check your inbox or spam folder to confirm your subscription.
Artificial intelligence - Organisation for Economic Co-operation and Development
Arrangements for the OECD's role as host will be finalised in the coming days. The GPAI will bring together experts from industry, government, civil society and academia to conduct research and pilot projects on AI. Its objective, as set out by founding members Australia, Canada, the European Union, France, Germany, India, Italy, Japan, Korea, Mexico, New Zealand, Singapore, Slovenia, the United Kingdom and the United States, is to bridge the gap between theory and practice on AI policy. An example would be looking at how AI could help societies respond to and recover from the Covid-19 crisis. Basing its Secretariat at the OECD will allow the GPAI to create a strong link between international policy development and technical discourse on AI, taking advantage of the OECD's expertise on AI policy and its leadership in setting out the first international standard for trustworthy AI – the OECD Principles on Artificial Intelligence.